Query representation by structured concept threads with application to interactive video retrieval

1047-3203/$ see front matter 2008 Elsevier Inc. A doi:10.1016/j.jvcir.2008.12.001 q This work is supported by the National Natural under the Grant Nos. 60621062 and 60605003, the N Projects under the Grant Nos. 2003CB317007, 2004C and the Basic Research Foundation of Tsinghua Nation Science and Technology (TNList). * Corresponding author. E-mail address: [email protected] (D. Wang). 1 Work performed while the author was at Tsinghua In this paper, we provide a new formulation for video queries as structured combination of concept threads, contributing to the general query-by-concept paradigm. Occupying a low-dimensional region in the concept space, concept thread defines a ranked list of video documents ordered by their combined concept predictions. This localized representation incorporates the previous concept based formulation as a special case and extends the restricted AND concept combination logic to a two-level concept inference network. We apply this new formulation to interactive video retrieval and utilize abundant feedback information to mine the latent semantic concept threads for answering complex query semantics. Simulative experiments which are conducted on two years’ TRECVID data sets with two sets of concept lexicons demonstrate the advantage of the proposed formulation. The proposed query formulation offers some 60% improvements over the simple browsing search baseline in nearly real time. It has clear advantages over c-tf-idf and achieves better results over the state-of-the-art online ordinal reranking approach. Meanwhile, it not only alleviates user’s workload significantly but also is robust to user mislabeling errors. 2008 Elsevier Inc. All rights reserved.